Neuromagnetic fields elicited by a bimodal input 
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1 Introduction 

In recent years noninvasive measurement such as 
magnetoencephalography (MEG), functional 
magnetic resonance imaging (f-MRI) and positron 
emission tomography (PET) has revealed many 
aspects of the brain function, from sensory 
projection to cognitive process. And most 
researchers in this field have studied brain responses 
to a single modal input. In order to well understand 
the working brain, however, it is necessary to study 
neural activities when a multi-modal input is applied. 
In the present paper we studied responses to a multi¬ 
modal stimulus using MEG in combination with 
MRI imaging. 

Responses to a single modal stimulus such as a light 
flash, a visual pattern and a sound have been 
reported in many literatures and their origins were 
estimated in the primary cortex and other areas by 
using MEG [1,2]. 

In the paper, in contrast, neuromagnetic responses 
were measured when a visual and an auditory 
signals were simultaneously given to a subject, and 
by analyzing spatio-temporal patterns of MEG, time 
course of activated regions in the brain and its 
relation to cognitive processes were investigated. 
Moreover, MEG responses were discussed in terms 
of subject’s attention. 

2 Methods 

Subjects were 4 adults (28 - 52 years of age) with 
normal hearing and vision. In a dark, magnetically 
shielded room, each subject was seated in a chair 
with eyes open. 

Visual and auditory inputs were simultaneously 
applied to a subject. The visual input was a flash of 
a light emission diode (LED) in the left visual field. 
The LED flashed randomly in either of 2 colors (red 
and green) for 20ms with random intervals between 
1500 - 2000 ms. Frequency of each color was 50%. 
Synchronized with the light flash, pure tone of either 
1kHz or 2kHz was randomly applied to the subject’s 
ears. Frequency of each tone was 50%, which was 
statistically independent of the light color 
randomization. 


The subject was instructed to pay attention to a 
specified combination of the light color and the tone 
pitch (called “target”) by mentally counting its 
occurrences. 

Neuromagnetic fields were measured by the 64- 
channel whole-cortex type MEG system (CTF Inc., 
Canada) with the onset of the bimodal stimulation 
being a trigger. They were detected for 1000 ms 
consisting of a 100 ms-pretrigger period and a 900 
ms-posttrigger period at 625 Hz-sampling rate. Their 
offsets were adjusted referring the 100 ms-pretrigger 
period and then they were band-pass filtered (0.01 - 
30 Hz) for noise reduction. For each combination of 
light and tone, 40 responses were averaged to obtain 
the event related field (ERF). Then, multidipole 
estimations were carried out based on the 
Grynszpan-Geselowitz’s equation. The location as 
well as the magnitude of the equivalent current 
dipoles (ECDs) was estimated during the period 
between 84.8 - 348.8 ms after the onset of the 
stimulus with 1.6 ms-step. The locations of the 
estimated dipoles were overlaid on the subject’s 
MRI which was T1 weighted to emphasize anatomy. 

3 Results 

Subjects correctly detected the targets in almost all 
experimental sessions. Figure 1 shows changes in 
mean ECD magnitudes in the period between 84.8 - 
288.0 ms for all subjects. Spatiotemporal 
distributions of the estimated ECDs during the 
period 84.8 - 348.8 ms after onset of the target 
stimulus are shown in Figure 2 for 2 subjects where 
the target was “green” and “1kHz”. 

There is a common pattern in those distributions. In 
the period about 90 - 130 ms, 3 ECDs were 
estimated in the right visual cortex and in both 
auditory cortices, respectively, throuout the period. 
Their magnitudes were 20 - 40 nAm. In the period 
about 130 - 150 ms, ECDs were estimated in basal 
ganglia (BsGa) and hippocampus (HP) as well as 
parietal lobe. The mean ECD magnitude was more 
than 50 nAm. In the period about 160 - 220 ms, 
ECDs were estimated mainly in parietal and 
temporal areas including angular gyrus (AG), 




Figure 1: Mean ECD magnitude (84.8- 288.0 ms). 
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Figure 2: Distribution of estimated ECD.s 

BsGa: basal ganglia, HP: hippocampus, ITG: inferior temporal gyrus, FG: fusiform gyrus, 
LG: lingual gyrus, CG: cingulate gyrus, IFG: inferior frontal gyrus, POr: pars orbitalis, 
MFG: middle frontal gyrus, SFG: superior frontal gyrus, PrG: precentral gyrus, PosG: 
postcentral gyrus, MTG: middle temporal gyrus, AG: angular gyrus, TGH: transverse 
temporal gyri of Heschl, STG: superior temporal gyrus, SMG: supramarginal gyrus, IPL: 
inferior parietal lobule, SPL: superior parietal lobule, PrCu: precuneus, OL: occipital lobe, 
R: right hemisphere, L: left hemisphere. 































supramarginal gyrus (SMG), inferior parietal lobule 
(IPL), postcentral gyrus (PosG), superior temporal 
gyrus (STG) and superior temporal sulcus, as well as 
visual and auditory cortices. Their magnitudes were 
10-40 nAm with the exception of Subject 2’s 
hippocampal activity (80 - 100 nAm) around 200 ms. 
In the period about 220 - 240 (250 for Subjects 3 
and 4) ms, ECDs were estimated again in basal 
ganglia. The mean magnitude was more than 40 
nAm. In the period about 240 - 300 ms, ECDs were 
estimated mainly in middle temporal gyrus (MTG), 
AG, PosG, cingulate gyrus (CG) and inferior 
temporal gyrus (ITG) as well as visual and auditory 
cortices. In addition, after about 320 ms, ECDs were 
frequently estimated in HP, BsGa and CG. 

In addition to the above-mentioned common 
features, ECDs in the frontal area were observed for 
all subjects, although they did not necessarily 
coincide in the time course among subjects. There 
were two major areas in the frontal lobe where 
ECDs distributed: (1) around the orbitofrontal area 
including a part of inferior frontal gyrus (IFG) and 
pars orbitalis (POr), and (2) around the dorsolateral 
prefrontal area including a part of superior frontal 
gyrus (SFG), middle frontal gyrus (MFG) and 
precentral gyrus (PrG). In about 200 -310 ms, ECDs 
around the orbitofrontal area were observed for 3 
subjects, while in about 160 -210 ms and about 300 
-320 ms for one subject. As to the ECDs around the 
dorsolateral prefrontal area, 2 subjects showed them 
in about 140 - 210 ms and 2 subjects in about 220 - 
330 ms. 

Figure 3 shows ERFs detected by some sensors over 
the right hemisphere for all combinations of light 
and tone where the combination of green light and 1 
kHz-tone was specified as the target. Comparing 
ERF for the target with that for the non-target which 
means the combination of red light and 2 kHz-tone 
where neither color nor pitch coincides with that of 
the target, the difference in the late component (after 
about 350 ms) was remarkable. In semi-target cases, 
where either color or pitch, but not both, coincides 
with that of the target, moderate differences from the 
non-target were observed in the late component. 

4 Discussion 

The results reveal some aspects of integration 
process of bimodal inputs in the brain. First, they 
show the time by which the integration is carried out 
and discrimination is made. As a matter of form, the 
experimental procedure in this study agrees with a 
paradigm known as the “odd-ball paradigm” in 
which some aspects of human response to 
environment such as novelty detection and attention 


paying can be physiologically observed [3], since 
target appearance was randomized with the 
frequency of 25% and the subject was required to 
pay attention to it by mental counting. The results 
show that the late component of ERF was prominent 
for the targets compared with non-targets and semi¬ 
targets in all possible combination of the visual and 
auditory stimuli, showing that the task can be 
practically be regarded as the odd-ball task and the 
late component of ERF corresponds to P300m which 
is usually observed in a single modal odd-ball task 
[4], One of the origins of P300m is reported to be in 
HP [5], The result that after about 300 ms ECDs 
were estimated in HP is consistent with the report. 
Therefore, it can be concluded that by this time 
(around 350 ms) the inputs through audition and 
vision would have been associated in the brain and 
discrimination whether it was the target or not 
would have been made. 

As to the early component (about 90 - 130 ms), the 
spatial pattern of ERF was well fitted by 3 ECDs. 
Their locations were estimated in the left and right 
auditory areas and the right visual area consistent 
with binaural hearing and hemifield (left) vision, 
showing that at this stage each input signal is 
processed in modality-specific cortical area. 

In about 130 -150 ms, mean magnitude of ECDs 
was more than 50 nAm for all subjects. This is 
mainly due to ECDs estimated in BsGa and HP. 
Similar activities were observed in 220 - 240 ms. 

In about 160 - 210 ms, activities in association 
cortices including parietal and temporal lobes 
prevailed. Then, in 220 - 240 ms, activity in BsGa 
was again observed, thereafter activities were 
observed in a wide variety of cortical area including 
frontal and inferior temporal areas as well as parietal 
and other temporal areas. From the fact that in our 
experiment activities in BsGa were observed in 
some restricted periods for all subjects and 
distribution pattern of ECDs changed after each 
period, it can be said that activities in BsGa would 
have some influences over the way of information 
processing in the brain [6], 

In addition, ECDs in the frontal area, especially 
around the orbitofrontal area and the dorsolateral 
prefrontal area are thought to be related to 
attentional process including deviance detection and 
working memory which is crucial in carrying out the 
experimental task. 

The activities in visual and auditory areas were 
intermittently observed through the whole period for 
all subjects, suggesting that during the period the 
sensory information would be frequently required 
and referred to. 
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Figure 3: Elicited magnetic fields over the right 
hemisphere for all combinations of light color and 
tone pitch (Subject 1). 

(1) Target: Green and 1 kHz. 

(2) Semi-target: Red and 1 kHz. 

(3) Semi-target: Green and 2 kHz. 

(4) Non-target: Red and 2 kHz. 









